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The management of energy-technology-innovation programs at the Department of Energy has a 
mixed record. There are several prominent “failures” such as the Synfuels and Clinch Breeder Reactor 
programs, and also a number of famous and notable successes. On balance, however, the National 
Research Council concluded in 2001 that, in the case of the fossil-fuel and energy-efficiency R&D programs 
the NRC study reviewed, the “benefits of these programs substantially exceed the programs’ costs and 
contribute to improvements in the economy, the environment, and national security” (NRC 2001, 63). 

Problems identified in the past with DOE's energy-technology-innovation activities include the 
narrowness of programs, inability to stop non-performing programs, poor coordination of activities across 
program boundaries (“stovepiping”), weak coordination of fundamental and applied RD&D, congressional 
earmarking, lack of clear leadership and authority, and limited technical skills among staff members. 
Significant progress has been made in recent years on many of these problems, particularly in the 
development of criteria for starting, continuing, and stopping projects; developing performance metrics for 
mid-stream review; soliciting external peer review; and establishing partnerships with the private sector and 
academia (including for cost-sharing arrangements). 

The energy-efficiency and renewable-energy (EERE) program, for example, recently put more than 
two-thirds of its R&D funding through competitive solicitations, terminated more than 60 projects, and 
conducted more than 20 external peer reviews (SPR 2002). While such progress is commendable, the 
Commission suggests that further management reforms are both possible and necessary. In what follows 
here, we review briefly the recommendations of recent studies that have addressed management issues, 
discuss some major successes and failures and the lessons learned from them, and then offer our own 
conclusions on DOE management of the federal government's energy-technology-innovation efforts. 

MANAGEMENT RECOMMENDATIONS OF PREVIOUS STUDIES 

In its 1997 report on Federal Energy Research and Development for the Challenges of the 21 st 
Century (PCAST 1997), the President's Committee of Advisors on Science and Technology addressed DOE 
management of energy R&D as well as budgets and opportunities. In 2001 the National Research Council 
likewise addressed management issues, along with its analysis of cost-benefit ratios, in Energy Research at 
DOE: Was It Worth It? (NRC 2001 ). In 2003, a task force of the Secretary of Energy’s Advisory Board 
released a report, The Future of Science Programs at the Department of Energy (SEAB 2003), which 
likewise addressed management as well as other issues. The management recommendations of all of 
these reports were generally consistent and mutually reinforcing, although differing in in emphasis in certain 
respects. 

The recommendations of these three reports are summarized in Figure 1 in a form that facilitates 
comparisons. Two common themes from the reports that deserve some elaboration here, before we turn to 
our own recommendations, are balancing of risks across an innovation portfolio and evaluating costs versus 
benefits of investments in energy-technology innovation. 

Balancing Risk in Energy Innovation Portfolios 

Managing innovation entails the management of a portfolio of investments in options at different 
stages of development, with different time horizons for potential commercialization, addressing different 
applications and challenges. Successes usually occur in the form of a few 'big hits' amidst many results 
whose value is uncertain at the time (and may turn out to be small, or even negligible) and some (sometimes 
spectacular) failures. Many R&D projects produce results that may appear to have little or no significance 



by themselves but are important because, when they are brought together with the results of other R&D 
efforts, they permit significant advances and even breakthroughs. A portfolio and its management should be 
judged in terms of the results from the whole portfolio over time. ..and the appropriate time may be long. 
Anecdotal lists of past failures tell us little nothing about the value of the overall portfolio. If one insists that 
there be no failures, one only ensures that there no risks will be taken and, accordingly, that no big 
successes will result. 

A common error in R&D management is the undertaking and continuing of projects and programs 
whose purpose (whether it be curiosity satisfaction, knowledge expansion, proof-of-concept, development, 
demonstration or test) is not really clear or is muddled by other motives. The challenge is to establish 
systems to encourage sound and transparent program assessments and feedback loops to ensure that 
lessons from prior successes and failures are taken into account. Ultimately, success will depend upon the 
skilled judgment of government officials. An important value of explicit decision criteria is to enable outside 
review and critique of ultimate decisions. It has been noted that, "The most important event in research is 
the prompt recognition of a dead horse, and its decent burial." 

This is true, but, unfortunately for managers trying to apply the dictum, many a research horse given 
up for dead has turned out to be a race winner in the hands of a persevering trainer. A famous example is 
Alexander Graham Bell’s telephone. Western Union rejected it because the company thought that anything 
that needed to be sent could be sent by telegraph, and, besides, no one would pay $5 just to talk to their 
grandmother in Indianapolis. It was only by an accident of publicity at the Chicago World’s Fair a European 
dignitary visited Bell’s booth and was interviewed there by reporters that set it on a useful road. There are 
failures of both kinds: going on too long and stopping too soon. Neither can be entirely prevented by even 
the best trainer; it’s the trainer's total track record that counts (Frosch 2004). 

Evaluating the Costs and Benefits of Past Efforts 

The 2001 NRC report concluded that, as might have been expected, a relatively small number of 
DOE research programs in energy efficiency and fossil energy accounted for the majority of the economic 
and environmental benefits (NRC 2001, p 63): 

"This characteristic of RD&D programs, in which a few “home runs” are responsible for the majority 
of returns on investments, is shared by industrial R&D programs and underscores the importance of 
maintaining a diversified portfolio of investments. The areas in which these benefits were greatest 
relative to program expenditures include residential and commercial construction, an industry that 
historically was not particularly innovative, and technologies to reduce environmentally harmful 
pollution. These are precisely the areas in which one would anticipate that public R&D programs 
are most likely to prove most effective. By contrast, DOE efforts to push the technology to 
commercial application in large, accelerated RD&D programs such as coal liquefaction have been 
extremely risky and prone to cost overruns and generally have yielded relatively small benefits 
relative to their high costs." 

The report estimated that total realized economic benefits associated with the energy-efficiency programs 
that it reviewed were in the $30 billion range, substantially exceeding the roughly $7 billion investment for 
energy-efficiency RD&D over the 22-year life of the programs (both in 1999 dollars). For fossil programs, 
the benefits exceeded costs in the 1986-2000 period (benefits of $7.4 billion and costs of $3.4 billion in 1999 
dollars), but costs exceeded benefits in the earlier 1978-1986 period (benefits of $3.4 billion compared with 
$6 billion in costs in 1999 dollars). 

We note that the period in which the DOE's fossil-energy programs "lost money" coincided with the 
unsuccessful SynFuels Corporation experiment. More importantly, perhaps, is that all of the NRC study's 
cost-benefit calculations were based on the highly conservative assumption (where "conservative" here 
means tending to understate the benefits) that all of the innovations resulting from DOE investments would 
have materialized from private-sector-funded efforts an average of five years later The NRC study itself 
emphasized, moreover, that "mechanical" cost-benefit analyses of R&D investments are inherently 
problematic, for several reasons: 



many of the benefits may be difficult to monetize (as is the case for many types of environmental 
benefits, for example); 


• benefits often will not have fully materialized at the time of any retrospective study; 

• R&D investments often yield results that have "option value" that cannot be accurately 
evaluated.without the benefit of a degree of predictive accuracy that no analyst can claim; 

• R&D investments - even those in efforts later judged as "failures" -- may contribute to the aggregate 
stock of scientific and engineering knowledge, with unknowable future benefits. 


PROMINENT FAILURES AND SUCCESS - AND LESSONS LEARNED 


Clinch River Breeder Reactor 1 

The Clinch River Breeder Reactor (CRBR) was announced by the Atomic Energy Commission in 
1972 as the nation’s first demonstration of liquid-metal fast breeder reactor plant. The CRBR was estimated 
to cost about half a billion dollars, with private industry paying for approximately 37 percent of the total cost. 
Justification for the CRBR was based on projected increases in the price of uranium fuel for the nation’s 
existing light water reactor fleet that would cause the price of nuclear electric power to become prohibitive. 
By “breeding” more fissionable fuel (plutonium) than it consumed, the pilot reactor would become the 
technological guarantor of economical nuclear electric power far into the future. 

At the project's initiation, however, the AEC’s own cost-benefit analysis was not favorable to the 
CRBR project as a commercial demonstration program. To get a positive net present value, the CRBR 
would have to be the first step leading to a large program of commercial breeder reactors. This outcome 
would require a very high rate of electricity demand, no competing technologies, and the disappearance of 
cheap uranium. The validity of these projects was soon called into question as the growth rate of electric 
power demand declined and cheap uranium did not disappear. With cheap uranium, there was no need for 
the CRBR project. By the late 1970s, controversy about the project had risen substantially, in part because 
of debates over the proliferation of nuclear weapons and the prospect of a domestic “plutonium economy.” 
Escalation of the CRBR costs led to even more controversy because by the end of the 1970s an additional 
$1 .7 billion was estimated to be required for the CRBR to achieve commercialization. The Senate killed the 
project in 1983 after $1 .6 billion had been spent, with an estimated cost to completion of at least another 
$2.5 billion. 

Lessons Learned: 

1 . The federal government should not be the primary source of funding for energy technology 
commercialization demonstration projects. Funding should be dominated by the potential industrial 
beneficiaries of the demonstrated technology. Massive federal funding of megaprojects galvanizes 
legislative, bureaucratic, and regional champions of the projects to a level beyond the point of 
productivity or economic justification and invites federal interference in project management. 

2. Before a project begins, the proposing industrial team must produce realistic cost, performance, and 
schedule estimates, including commitment to its portion (majority) of the cost of the project. These 
estimates must be reviewed by an independent and knowledgeable team before project approval. 

3. Before a project begins, clear mutually agreed to technical cost, performance, and schedule goals 
must be established, along with sound criteria for changing or canceling the project if reasonable 
progress toward those goals is not met. 


1 Excerpted from PCAST 1999, pg. 2-17. 



4. As a corollary to number 3, an oversight process should be established to provide a periodic 
independent evaluation of project management, performance, schedule, and cost control. 

5. Although federally funded projects cannot be insulated from political interference and “second- 
guessing,” the government should resist making politically determined decisions that compromise 
the justified continuation or cancellation of energy projects. 

Magnetohydrodynamics 2 

This program in the fossil energy division of DOE was targeted at using electromagnetic induction to 
produce electric power from coal. After the creation of DOE in 1977, MHD quickly became one of its major 
technology programs. DOE’s MHD program was focused on the development of two major test facilities: the 
Component Development and Integration Facility (CDIF) and the Coal-Fired Flow Facility (CFFF). Between 
1978 and 1993, DOE expenditures for the MHD program totaled $1 billion (1999 dollars). Over this same 
period, private industry cost sharing totaled about $61 million. Cost sharing began in 1986 when private 
industry was required by legislation to cost share, initially at 10 percent, but increasing to 35 percent by the 
end of the proof-of-concept program in 1993. Almost half of the DOE expenditures for MHD R&D occurred 
in the first 4 years from 1978 to 1981, during design and construction of the test facilities. After 1982, Doe 
did not request any funds for the MHD program from 1982 to 1993, when the program was finally 
terminated. In those years, the funding came from direct congressional line item additions to the DOE 
budget. While the MHD program was modestly successful in the proof-of-concept phases, system 
evaluation studies were indicating that the cost to design, construct, and operate a central station MHD 
power generation facility was much higher than the corresponding costs for other coal-fired power 
generation options. In addition to the high costs, the claim for high-cycle efficiencies was questionable. This 
raised real doubts that the MHD system could compete on an efficiency basis with the advanced gas turbine 
combined cycles used by the utility industry. These doubts ultimately led to program termination. 

Lessons Learned: 

1. Substantial funds continued to be spent after 1981 to prove the MHD concept in the face of data that 
were showing significant technical barriers to the successful development of the concept. At the 
same time, studies indicated that even if developed, the MHD power generation system would not 
be competitive on an efficiency or cost basis with alternatives that were already in use by the utility 
industry. The data suggest that this information led to DOE’s decision not to request funding after 
1981, except for 1985. Congress continued to fund this program through 1993, however, an 
indication of the strength of congressional support for MHD. 

2. Private sector interest in developing a technology, as evidenced by a willingness to cost share in the 
demonstration process, must be considered. There was no cost-sharing in the design, construction, 
and early operation of the costly large-scale facilities. 

3. Difficult decisions to terminate programs must be made as early as possible and available funds 
redirected to the areas of greatest potential. 

Advanced Efficient Refrigeration Program 3 

In 1977, DOE initiated an appliance product development program that included emphasis on refrigerator- 
freezers and supermarket refrigeration systems. Manufacturer involvement was substantial from the outset. 
DOE targeted both improved components, starting with the electricity-intensive refrigerator compressor, and 
computer tools for analyzing refrigerator design options. Early successes included a compressor system 
that achieved 44 percent efficiency improvement over the technology commonly used in refrigerators of the 
late 1970s. The total funding from 1978-1981 for refrigerator compressor R&D was $1.56 million (1999 
dollars). The research was cost-shared with industry through a competitive solicitation. The winning 
contractor, Columbus Products Company, contributed $550,000 (1999 dollars). Since then, the average 
electricity consumption of refrigerators has been reduced by more than two-thirds, even as average unit 
sizes increased, performance improved, and ozone-depleting substances were removed. DOE was an early 


2 Excerpted from NRC 2001, pg. 190-193. 

3 Excerpted from NRC 2001, pg. 95-99. 



and effective leader starting with its 1977 launch of product development. DOE’s initial investment helped 
demonstrate that a full-featured refrigerator could use 60 percent less electricity than comparable 
conventional units. These successes strongly influenced the enactment of increasingly demanding 
efficiency standards, first in California and ultimately by DOE itself, under authority of the National Appliance 
Energy Conservation Act of 1987. Total economic benefits deriving from this program are estimated at $7 
billion (1999 dollars). Other benefits include substantial emissions reductions, reduction in energy 
consumption, improved electric system reliability, and knowledge benefits related to system optimization, 
R&D on energy-saving components and features, and application to air conditioners. 

Lessons Learned: 

1 . This case study underscores the value to society of integrating RD&D and minimum efficiency 
standards as an instrument for accelerating technological innovation. 

2. Competitive solicitation for cost-sharing can be very useful in public-private partnerships. 

Oil and Gas Drilling, Completion, and Stimulation Program 4 

This program began as the drilling research program initiated in 1975 following the Arab oil embargo. The 
program focused on developing drilling technology to increase domestic oil and gas production. In 1993 it 
was separated into oil and gas subsections. The current program is targeted at the development of 
technology to reduce drilling costs, minimize formation damage, reduce environmental risks, reduce surface 
footprint of onshore and offshore drilling, and improve access to culturally and environmentally sensitive 
areas. The program has consisted of a very large number of small projects covering almost every facet of 
the various drilling, completion, and stimulation technologies. Historically, much of the technology is utilized 
in the oil industry by service companies. The technologies are developed primarily by the oil service 
companies and oil companies. Industry has indicated its interest in these programs by funding 29 percent of 
the total expenditures from 1978 to 1999. The cumulative cost to DOE of the oil program from 1978 to 1999 
was $48 million and the cumulative cost of the gas program was $31 million, for a total of $79 million. The 
economic benefits from the oil programs alone are assessed at $2.2 billion. The total economic value of the 
gas program has not been assessed, but two projects alone (underbalanced drilling technologies and high- 
temperature measurement while drilling/logging while drilling) realized a benefit of $252 million. Significant 
environmental benefits have accrued from this program including smaller footprints, reduced noise, lower 
toxicity of discharges, reduced fuel use, and better protection of sensitive environments. Since these 
programs are all directed at increasing the production of oil and gas in the United States, they directly 
contribute to national security. 

Lessons Learned: 

1. Many small and medium-size firms (such as those in the oil service industry) have limited R&D 
budgets so government can play an effective role with a relatively small investment in high risk 
projects to stimulate advances in technologies that can have large positive impact on the industry 
and benefit the nation. Also, in the oil industry, technology disperses quickly, making it difficult to 
capitalize on R&D investments without government assistant. 

2. The key to an effective program is interaction with and feedback from the industry. 

Commission Assessment 
Management Improvements at DOE 

Considerable effort has been devoted to implementing the recommendations from the NRC and 
PCAST and other assessments. In particular, progress has been made in: 

• Creating explicit goals and strategic plans for the programs within the broad divisions of Fossil 
Energy and Energy Efficiency and Renewable Energy; 

• Establishing criteria for investment, ongoing performance review, and disinvestment; and 


4 Excerpted from the NRC 2001 study, pg. 193-198. 



Creating mechanisms for internal and external peer review and advice. 


Remaining challenges relate primarily to 

• better linking of 'stovepipes' in DOE's Divisions and Sections; 

• truly coordinating the energy innovation programs of the Department at a high-level; 

• setting up stronger systems for review and evaluation; 

• compelling more coordination between fundamental and applied energy technology research; 

• increasing the investment in science and technology training of current and future staff; and 

• fostering more cooperation with other government agencies to make sure that R&D programs are 
consistent with energy-technology deployment policies. 

Much of what is required in these respects will follow if DOE extends and re-energizes its efforts to 
implement the recommendations of the earlier studies summarized above in Figure 1 . 

It is, of course, up to the Secretary and the Congress to organize and manage the Department. 
Recognizing the heavy burden carried by the Deputy Secretary for Energy, the Commission encourages the 
Secretary and Congress to consider establishing an Under Secretary for each of the core missions of the 
department: nuclear security, nuclear waste management, energy innovation and policy, and management. 
An organization diagram incorporating these suggestions is depicted in Fig. 2. Grouping all of the energy 
innovation activities together is intended to enhance overall coordination as recommended by all three 
recent reports. 5 Note that in this new organizational formulation, the offices of counterintelligence, 
intelligence, and security report directly to the Secretary, as proposed to Congress by the Bush 
Administration earlier this year. 

Congressional Earmarking 

Although Congress is unquestionably responsible for appropriations, the current practice of non-competitive 
earmarks that bear little connection to the research goals or plans is harmful and raises questions regarding 
the public value of funds spent for energy innovation. So-called 'pork-barrel' projects are likely to be 
undertaken for the wrong (or unclear) reasons, in the wrong place, by the wrong people, with results that are 
often, at best, unclear. For some energy R&D programs, half of their budget is being earmarked. The 
Commission recommends that Congressional Leadership seek to ensure that earmarks are consistent with 
the specific strategic objectives of the affected program. 
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Figure 1 

Category 

PC AST 1997, Federal Energy Research 
and Development for the Challenges of the 
21 st Century (pgs. 7-24 to 7-33) 

NRC 2001, Energy Research at DOE: 
Was It Worth It? (pgs. 62-69) 

SEAB 2003, Critical Choices: Science , Energy , 
and Security. Task Force on The Future of 

Science Programs at the Department of Energy 
(pgs. 18-25) 

Leadership and 
Organizational Structure 

One person should be in charge of energy 
innovation activities at the Under Secretary 
level 

Need for more flexibility in the structure 
of programs 

DOE should have an Undersecretary for Science to 
serve as steward for basic research and connect with 
applied research programs 

Strategic Planning 

Create a long-term strategic plan and vision, 
then define programs, create partnerships, 
etc. 

Develop clear performance targets and 
milestones, including intermediate 
performance targets (go/no-go criteria) 

Focus on outcomes; more merit-based competition; 
embark on 3 major, highly- visible research initiatives 
to promote leadership 

Review and Evaluation 

Review and evaluation are crucial; minimize 
inside agency and maximize use of outside 
advisory bodies. Figure out how to terminate 
consistently unpromising projects (in PC AST 
1999) 

Main recommendation of study: Develop 
and adopt analytic framework to better 
evaluate existing programs uniformly. 
Requires better data collection. 

Use merit-based decision making 

Partnerships 

Cultivate more and stronger partnerships 
with firms, labs, and universities 

Continue cost-sharing projects with 
industry and other partners; try to 
quantify costs that are shared 

More partnerships with industry and academia 

Technical Oversight 
Committees 

Use technical oversight committees for each 
program (including outside experts) 

Ongoing evaluation relies on regular peer 
review by panels of outside technical 
experts 

Regular reviews are necessary 

External Peer Review 

Do more external peer review of overall 
strategy and plans 

Regular external review is needed, 
particularly regarding application of 
analytic framework 

Undersecretary for Science should have high-level 
Science Advisory Board 

Coordination Within DOE 

More coordination between fundamental and 
applied technology RD&D through co- 
funding and co-management of basic and 
applied research More coordination among 
technical programs (less “stovepiping”) 

More coordination with other policies 
that might promote deployment of 
advanced technologies 

Science programs have historic reputation of being 
badly managed, fragmented and politically 
unresponsive 

Coordination Outside DOE 


More coordination with other 
government agencies, esp. EPA 


Portfolio Management 

Use strategic criteria to determine DOE 
energy technology portfolio (economic, 
enviro., security, diversity) 

Allocation of RD&D funds requires the 
application of priorities based on stated 
policy as well as the exercise of judgment 
rather than mechanical application of 
cost-benefit techniques 

Focus on three areas: energy production, storage, 
distribution, management; adv. Computation and 
simulation for basic science; frontier research facility 
for basic science 

Role of Congress 

Many management problems begin with 
congressional micromanagement of programs, 
earmarking, and dramatic shifts in budget 
levels and directives 


Congress has not given DOE science budgets the 
priority merited by their importance; improve 
outreach capability to Congress, public, and other 
agencies 

Staffing and Technical 

Skills 

DOE staff technical skills should be 
strengthened through training, targeted 
hiring, and through staff rotations. 


More federal invest, in physical sciences and adv. 
engineer.; DOE should work to educating future 
scientists and engineers for careers in DOE-related 
fields (K-12, undergrad through post-co) 

Deferred Maintenance 



Program should be established to renew labs because 
facilities and infrastructure are in a state of decay. 



Figure 2: Proposed Organizational Changes to the U.S. Department of Energy 
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